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Abstract 


The purpose of this report is to describe a science competency model and 3 related learning 
progressions, which were developed by applying the CBAL™ approach (Bennett & Gitomer, 
2009) to the domain of middle school science. The Cognitively Based Assessment of for, and as 
Learning (CBAL) science competency model and its related learning progressions were 
developed by reviewing existing literature on learning sciences and science education, which 
have placed increasing emphasis on learners’ knowledge and ability to apply scientific 
knowledge to conduct evidence-based reasoning. In this report, we present the 5 competencies in 
our science competency model that reflect current efforts in the Next Generation Science 
Standards and the recent reform-based curriculum to promote integrated and generative 
understanding. In addition, we report 3 hypothesized learning progressions related to our 
competency model to define the increasing sophistication of both content understanding and the 
capacity to carry out scientific inquiry. Then we discuss features of assessment prototypes 
developed under the guidance of the competency model and the learning progressions, by 
illustrating parts of 1 sample formative assessment task prototype. 

Key words: science competency, competency model, learning progressions, cognitively based 
assessment, fonnative assessment, summative assessment 
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The use of assessments is critical to educators to both inform and guide instruction. The 
use of a wide variety of assessment tools can enable teachers to determine where students are in 
their conceptual development and where they might need additional support or whether the 
instructional strategies in use need modification. In this way, assessments can be used to improve 
classroom practice, plan the curriculum, and reflect on one’s own teaching practice. This 
reported project is part of the effort of the CBAL™ (Cognitively Based Assessment of, for, and 
as Learning) initiative, a model for an innovative K-12 assessment system that integrates 
learning-sciences theories with content standards. The model measures achievement through 
periodic summative assessments that incorporate computer-delivered, scenario-based tasks. 
Change in students’ thinking and abilities, as well as teachers’ practices, are postulated to occur 
through the use of CBAL competency models, summative assessment, classroom formative 
assessment, and professional support. The assessment system is intended not only to measure 
student achievement but also to facilitate it (Bennett, 2010; Bennett & Gitomer, 2009). 

Our competency model of science reflects current insight from the cognitive and learning 
sciences research. We believe any science assessment that focuses predominantly on the facts of 
science fails to provide a rich picture of what proficiency in science looks like. Reformers in 
science education have advocated that students proficient in science should be able to inquire, to 
reason scientifically, to apply science concepts to real-world situations, and to communicate 
effectively what students know about science (Pellegrino, Chudowsky, & Glaser, 2001). 
Therefore, assessment of scientific concepts and theories must be focused not only on measuring 
knowledge of subject matter, but on how relevant that knowledge is in building the capacity to 
apply scientific principles on a daily basis. 

The purpose of this report is to describe a science competency model and its related 
learning progressions, which were developed by applying the CBAL approach (Bennett & 
Gitomer, 2009) to the domain of middle school science. The development is consistent with the 
construct modeling approach (Brown & Wilson, 2011; Shin, Stevens, & Krajcik, 2010), which 
stresses developing assessments and instruments that are deeply rooted in a specific model of 
cognition, one in which “the model of cognition is defined by one or more proficiencies 
represented by unbound continuous latent variables” (Brown & Wilson, 2011, p. 225). Our 
competency model documents the continuous latent variables in the domain of middle school 
science. In addition, in a construct modeling approach, learning is conceptualized as improved 
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competence towards increasingly higher levels of complexity and sophistication, rather than the 
greater accumulation of information and skills. Therefore, we are also developing hypothesized 
learning progressions related to our competency model to define the increasing sophistication of 
both content understanding and the capacity to carry out scientific inquiry. These hypothesized 
learning progressions describe the nature of student thinking by including nonnonnative ideas or 
misconceptions that students may have. The goal of using learning progressions in assessment is 
to characterize student thinking in ways that can be used for instructional decision making, rather 
than just indicating whether students appear to know or not know a given science concept. 
Therefore, in our project, the purpose of developing hypothesized learning progressions is to 
design assessment opportunities to gather evidence of where students fall on the learning 
progressions and use such evidence to inform future instruction. In the following sections, we 
describe in detail our competency model and its related hypothesized learning progressions. 

Then we present a formative assessment prototype developed under the guidance of the 
competency model and the learning progressions. 

The Competency Model of Middle School Science: Defining and Developing the Constructs 

The CBAL science competency model (Figure 1) and its related learning progressions 
were developed by reviewing existing literature on learning sciences and science education, 
which have placed increasing emphasis on learners’ developing conceptual knowledge and their 
ability to apply their scientific knowledge in order to reason with evidence. Our competency 
model for science includes five general dimensions: core ideas (i.e., subdisciplinary conceptual 
knowledge), science practices (i.e., knowledge of how to produce knowledge), crosscutting 
concepts (i.e., concepts that bridge disciplinary boundaries and provide organizational 
frameworks for connecting ideas), epistemic beliefs and metacognition (i.e., beliefs regarding 
knowledge and knowing, and strategies of knowing about knowing), and motivation (i.e., values, 
interest, and engagement). Our competency model is informed by and is consistent with research 
on teaching and learning (Duschl, Schweingruber, & Shouse, 2007), as well as current 
frameworks informing standards for science education in the United States, such as the National 
Research Council (NRC) Framework for K-12 Science Education (NRC, 2012) and the Next 
Generation Science Standards (NGSS, 2013). Note that, with our current prototype tasks, we 
intend to measure only three of these five dimensions: core ideas, science practices, and 
crosscutting concepts. To be proficient in science, students need to demonstrate their ability to 
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simultaneously apply the three interconnected competencies (represented as solid lines in Figure 
1) to address scientific questions or problems. However, the other two dimensions, epistemic 
beliefs/metacognition and motivation, will be an important part of the design of the task. Later 
we may choose to measure these relevant, but hard-to-measure general competencies. Epistemic 
and emotional aspects are important in science learning because they have mediating effects on 
such internal psychological processes as acquisition of knowledge and skills (represented as 
dotted lines in Figure 1), and, thus, will have direct and indirect impact on the validity of 
assessments. 



Figure 1. CBAL science competency model. 

Our view of developing proficiency in science builds on the four elements of learning 
in science as described in the NRC framework (NRC, 2012): conceptual development, which 
involves building connections between different ideas or reconceptualizing ideas in science 
(e.g., all matter, no matter how small or in what state, has mass and volume); cognitive 
development, which involves developing the ability to reason critically in a scientific manner 
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(e.g., understanding the relationship between cause and effect or supporting claims with 
scientific evidence); epistemic development, which involves developing ideas about science 
(e.g., understanding how we know what we know) and the processes, values, and implications 
of scientific knowledge; and social and affective development, which involves developing the 
ability to work collaboratively and to offer an engaging and stimulating experience (Osborne, 
2007). These four elements of learning in science map onto the four strands of science 
proficiency defined by the comprehensive report on K-8 learning and teaching, Taking 
Science to School : (a) know and interpret scientific explanations of the natural world, (b) 
generate and evaluate scientific evidence and explanations, (c) understand the nature and 
development of scientific knowledge, and (d) participate productively in scientific practices 
and discourse (Duschl et al., 2007; Osborne, 2007). These strands reflect the proficiencies of 
applying conceptual knowledge structures and using those structures to predict, explain, or 
model phenomena (the conceptual); constructing knowledge by generating evidence through 
the process of scientific inquiry—which ultimately leads to the generation and evaluation of 
evidence to revise and refine explanations or models (the cognitive); reflecting on the nature 
of science and the status of one’s own knowledge (the epistemic and metacognitive); and 
collaborating and participating in the discourse and norms of science with the motivation to 
sustain deep learning (the social and affective). 

In summary, the five dimensions in our science competency model reflect current efforts 
in the NGSS and recent reform-based curriculum (Krajcik & Reiser, 2004) to promote integrated 
and generative understanding. First, when learning about the core ideas in science, students need 
to develop connections between explanatory models of scientific phenomena in their daily life 
and the conceptual understanding of scientific knowledge. Second, through the essential science 
practices such as making observations, developing hypotheses, designing and conducting 
investigations, and developing evidence-based explanatory models, students can develop the 
conceptual, cognitive, metacognitive, and epistemic knowledge, skills, and abilities to 
understand the natural world (Bransford, Brown, & Cocking, 2000; Duschl et al., 2007). These 
practices often take place in collaborative learning environments that foster social and affective 
development (Jones & Issroff, 2004). Third, understanding the crosscutting concepts in science 
provides students with an organizational framework to connect various disciplinary ideas and 
consequently build a coherent view of the world. These are the three competencies (core ideas, 


4 



science practices, and crosscutting concepts) that we intend to measure in our CBAL 
assessments. In addition, the other two competencies in our model (epistemic 
beliefs/metacognition and motivation) are important facets that also influence learning. They 
both are connected closely with other competencies in our model. If science assessments do not 
incorporate a constructivist epistemic perspective, they are likely to measure only fixed factual 
knowledge, which does not provide an adequate picture of what and how students learn. In 
addition, if science assessments do not consider students’ alternative concepts and how they 
learned scientific knowledge, the science assessments will not provide opportunities for students 
to build on their prior knowledge to develop new knowledge. Finally, the students’ motivation 
has a direct impact on their achievement (Organisation for Economic Co-operation and 
Development [OECD], 2004; T. Stevens, Olivarez, & Hamman, 2006; Stipek et al., 1998; Zhu & 
Leung, 2011). Therefore, it is critical to consider these five dimensions in the design of 
assessments. 

In our competency model, we identify the candidate core ideas, which are categorized 
into three broad domains of physical science, life science, and earth and space science. The 
science practices include five central practices: constructing explanations, constructing 
arguments, developing and using models, engaging in empirical investigations, and collaborating 
and communicating. The crosscutting concepts that we focus on include systems thinking, cause 
and effect, and conservation of matter and energy. In the following sections, we describe specific 
definitions of each competency in the model and their importance to becoming proficient in 
science. 

Dimension 1: The Core Ideas 

To avoid targeting conceptual knowledge that is too superficial and reveals little about 
student learning in our assessments, we have selected a set of foundational core ideas that allow 
us to explore the sophistication of students’ conceptual knowledge and scientific reasoning. One 
rationale for organizing content around core ideas comes from research that compares the 
knowledge and problem-solving abilities of experts and novices in certain fields. This research 
shows that experts tend to use core principles from their discipline to make sense of new 
information or tackle novel problems, while novices tend to hold isolated and even contradictory 
pieces of knowledge that are not organized around core principles of the discipline (Bransford et 
al., 2000). Therefore, to help students shift from novice to expert status, it is essential to help 
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students learn these core ideas by engaging them in the practices of science. The importance of 
focusing on core ideas has not been lost on those involved in current reform efforts in science 
education and, consequently, new frameworks for standards that have been developed—such as 
A Framework for K-12 Science Education (NRC, 2012) and the NGSS (2013). 

Here, we identify the candidate core ideas that we will target in CBAL science over the 
next few years, which are categorized into three broad domains covered in the K-12 standards as 
described in current reform documents (e.g., National Assessment Governing Board [NAGB], 
2009; NRC, 2012): physical science, life science, and earth and space science. In physical 
science, we plan to target three foundational core ideas: matter, energy, and forces. In life 
science, we plan to target two foundational core ideas: ecosystems and structures and processes. 
In earth and space science, we plan to target one foundational core idea: earth systems. Note that 
each of these core ideas includes subcomponent big ideas (e.g., for matter, there are two big 
ideas: properties and structure of matter and changes in matter). The reasons for selecting this 
limited set of core ideas to target in CBAL over the near term are that there are clear and visible 
connections among these different core ideas that students should be developing and that should 
be apparent to teachers. 

In terms of the assessment prototype development, we are starting with one core idea in 
physical science: matter. It is particularly important to start with the core idea of matter because 
it is one of the most foundational and generative ideas in all of science, in particular the atomic- 
molecular theory of matter. In support of this point, according to Nobel laureate physicist 
Richard Feynman, if all the world’s scientific knowledge were destroyed and he could only 
salvage one idea to pass on to future generations, he would keep the atomic-molecular theory 
(Smith, Wiser, Anderson, & Krajcik, 2006). 

Dimension 2: The Practices of Science 

Our CBAL science competency model includes five central practices that are aligned 
with the NRC framework for K-12 science education (NRC, 2012). These practices are 
constructing explanations, constructing arguments, developing and using models, engaging in 
empirical investigations, and collaborating and communicating. In this section, we describe each 
of these five central practices by defining the construct and elaborating its importance in learning 
about science. 
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Science Practice 1: Constructing scientific explanations. Scientific explanations 
attempt to answer three questions: what we know (the ontological question), why it happens (the 
causal question), and how we know (the epistemic question, Osborne & Patterson, 2011). A 
scientific explanation includes three components: (a) a claim, (b) evidence, and (c) reasoning 
(McNeill, Lizotte, Krajcik, & Marx, 2006). Therefore, a claim itself is not a complete 
explanation because it lacks evidence and reasoning. These three components are further 
elaborated below: 

• Clainv. The claim makes an assertion or conclusion that addresses the original 
question or problem about a phenomenon. 

• Evidence : The data from an investigation supports the student’s claim. This data can 
come from an investigation that students complete or from another source, such as 
observations, reading materials, archived data, or from theoretical sources such as 
computer simulations. Data need to be appropriate, sufficient, accurate, and precise 
to support the claim. By appropriate, we mean data that are relevant to the problem 
and help detennine and support the claim. Sufficient refers to providing enough data 
or other kinds of evidence to convince another individual of the claim. Often 
providing sufficient evidence requires using multiple pieces of data. By accuracy, we 
mean the degree of closeness of measurements of a quantity to that quantity’s true 
value. By precision, we mean the degree to which repeated measurements under 
unchanged conditions show the same results. 

Different models of scientific explanations may include different kinds of evidence. 
Braaten and Windschitl (2011) summarized five major models of scientific 
explanations from the philosophy of science literature that are commonly used in 
science education. Three types of evidence derive from these explanation models. The 
first type of evidence is a natural law-like statement that fosters logical reasoning. 
Students’ first attempts at explanations often follow this fonn. When students use this 
type of evidence, it is possible that the reasoning is logical but the evidence is 
irrelevant or based on students’ naive or unstable knowledge. Due to the unstable 
nature of the reasoning criteria that students apply, the explanation produced is not 
robust. The second type of evidence is a trend or pattern in data from observations or 
empirical investigations that can be used to make statistical inferences. The third type 
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of evidence is that of a major scientific theory that can be applied to make sense of 
observed phenomena. 

• Reasoning : The reasoning li nk s the claim and evidence and shows why the data count 
as evidence to support the claim. Often, in order to make this link, students must 
apply appropriate scientific principles or theories. 

The process of constructing effective scientific explanations is considered an effective 
approach to constructing knowledge as it moves beyond descriptions of observable natural 
phenomena into theoretical accounts of how phenomena unfold the way they do (Braaten & 
Windschitl, 2011; Chi, 2000; Chi & Bassok, 1989). The researchers observed that 
undergraduates who demonstrate proficiency in problem solving often tend to produce more self¬ 
explanations, which are defined as “content-relevant articulation uttered by the student after 
reading a line of text” (Chi, 2000, p. 165). In addition, a number of researchers developing 
curriculum materials at the middle school level have found engaging students in developing 
written explanations is a useful and valid assessment strategy to probe student thinking, as well 
as an effective strategy to support learning by encouraging students to make connections 
between evidence and theory—and, therefore, generate cognitive connections (Lombrozo, 2006; 
McNeill et ah, 2006; Strevens, 2006; Trout, 2007). Finally scientific explanations often require 
students to apply deep principles or scientific theories to account for natural phenomena. 

Unfortunately, students have little experience constructing explanations, nor have they 
been exposed to the explanation framework above—even fewer have been asked to reflect on the 
nature of explanation in science—therefore, to assess explanations and students’ ability to 
construct explanations, it might be necessary to provide scaffolds for students during formative 
assessment and later assess students without the scaffolds. One such scaffold is to make the 
explanation framework explicit to students. In addition, researchers have shown that the process 
of evidence evaluation is very messy and requires an understanding of error and variation. In 
particular, Kuhn and her colleagues (Kuhn et ah, 1988) found that students have a variety of 
strategies for keeping theory and evidence in alignment with one another when they were, in fact, 
discrepant. One tendency is to ignore, distort, or selectively attend to evidence that is inconsistent 
with a favored theory. Therefore, it is important to help students understand what is counted as 
appropriate, sufficient, accurate, and precise evidence that can support or reject a claim. One 
possible strategy is to give examples of how to answer the explanatory questions using prompts 
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such as “My evidence is consistent with ... and thus supports my claim because ...” or “I believe 
my evidence is valid and reliable because ...” 

Science Practice 2: Constructing scientific argumentation. Scientific argumentation is 
a social and collaborative process that is at the heart of scientific inquiry (Duschl & Osborne, 
2002). It examines the question of whether the explanation is valid—that is, whether it succeeds 
in generating understanding and whether it is better than competing accounts (Osborne & 
Patterson, 2011). Argumentation includes any dialog that addresses “the coordination of 
evidence and theory to support or refute an explanatory conclusion, model, or prediction” 
(Osborne, Erduran, & Simon, 2004, p. 995). According to Walton (1989), there are two goals of 
argumentation: The first is to secure commitments from the opponent that can be used to support 
one’s own argument; the second is to counter an opponent’s position by identifying and 
challenging weakness in the opponent’s argument. Argumentation often consists of three major 
components: (a) claims, (b) grounds, and (c) rebuttal (Osborne et ah, 2004; Toulmin, 1958). 

• Claims'. Claims are statements in the form of declarative sentences that answer the 
controversial questions. 

• Grounds'. Grounds include data, warrants, and backings. 

• Rebuttal : Rebuttals include attacks on the grounds of a claim or attacks directly on a 
claim. 

We intentionally treat argumentation and explanation as distinct practices. Consistent 
with Berland and Reiser (2008), we believe the practices of explanation and argumentation are 
complementary. Explanation provides a product around which the argumentation can occur. 
However, argumentation creates a context in which robust explanations are valued. Osborne and 
Patterson (2011) also attempted to make a rigorous distinction between these two practices. They 
pointed out that a “defining feature of an explanation is that the phenomenon to be explained is 
not in doubt” (p. 629), while “there is always a substantial degree of tentativeness associated 
with any argument” (p. 629). Although, in our competency model, we are not making this 
rigorous distinction between explanation and argumentation, we agree that argumentation 
focuses more on the tentative nature of science and explanation focuses more on the power of 
using evidence to interpret a scientific phenomenon. Therefore, explanation and argumentation 
are two closely related, but distinct, practices. We agree with Osborne and Patterson that one 
major purpose of argumentation is to judge the coherence, plausibility, and comprehensiveness 
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of alternative explanations. However, we disagree that a statement about scientific phenomenon 
cannot serve as a claim in the process of constructing an explanation. In our view, a claim can be 
a statement that describes a scientific phenomenon, a statement that accounts for a tentative 
scientific theory, or a prediction that needs to be confirmed. For example, students can argue 
about the following claim: sugar does not cease to exist when it gets dissolved in the water. In 
this example, the claim is a true statement that is not tentative. However, it is still debatable for 
students because of the misconceptions or nonnormative ideas that they may hold. Based on 
these assumptions, we apply the claim, evidence, and reasoning framework to practices of 
constructing explanations and practices of constructing arguments. Nevertheless, there are 
several significant distinctions between explanation and argumentation. In argumentation, it is 
important to include rebuttals. However, the context of an argument task is open-ended and does 
not suggest an absolute correct or incorrect answer. Another key difference is that argumentation 
moves to interchanges in a social context; therefore, when students engage in argumentation 
activities, they may find themselves defending, debating, or critiquing products or ideas of their 
peers. 

Argumentation is not only a critical part of science, it also supports science learning and, 
therefore, we must assess students’ ability to engage in argumentation. Learning is viewed by 
many as having a social dimension; studies have shown that students often construct meaning 
through interaction with their peers or other community members (Jones & Issroff, 2004). 

Studies have shown that students not only have to learn to develop valid arguments, but they will 
also leam science while arguing (e.g., Erduran, Simon, & Osborne, 2004; Jimenez-Aleixandre & 
Pereiro-Munhoz, 2002; Osborne, Erduran, Simon, & Monk, 2001). Other studies have shown 
that when students debate other students’ ideas publicly, requiring them to verbalize and defend 
their own ideas, or to identify flaws in others’ ideas, this interchange can promote sense-making 
(Bransford et ah, 2000; Duschl et ah, 2007; Michaels, Shouse, & Schweingruber, 2008). 

Unfortunately, students in most classrooms have little experience engaging in 
argumentation. Thus, assessments that attempt to probe students’ ability in this area are likely to 
require multiple opportunities for students to encounter alternative ideas and a social context to 
communicate ideas. In assessment settings, particularly assessments taken by individual students, 
one possible approach to assess students’ argumentation ability is to present a student with 
multiple alternative opinions from virtual students and ask the student to critique them. Another 
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more authentic approach is to log the dialogue or interactions between students when they 
engage in critiquing or defending each other’s ideas and then score those conversations or 
interactions. 

Science Practice 3: Developing and using models. The value of scientific models and 
modeling has been increasingly recognized among the science education reform movements 
(American Association for the Advancement of Science, 1993; Duschl et ah, 2007; NRC, 1996; 
National Science Board Commission on Precollege Education in Mathematics Science and 
Technology, 1983) because scientific models are useful tools for formulating hypotheses to be 
tested and for describing scientific phenomena (Gilbert, 1995). Scientists use models to represent 
their understanding of a phenomenon, including facilitating the development of research 
questions, explanations, predictions, and communications with others. A scientific model is an 
abstraction or a simplification of a system that makes its central features explicit and visible 
(Harrison & Treagust, 2000). 

In science education, the tenn model is typically used in two related ways (Gobert & 
Buckley, 2000)/ First, a mental model refers to an individual’s internal representation, or that 
individual’s understanding of a phenomenon, which is often implicit to people. Second, a 
conceptual model is an explicit external representation (such as a diagram or computer 
simulation) of the mental model, which is often analogous to the phenomena of interest. Such 
mental models and conceptual models are the product of scientific modeling. Therefore, we 
define scientific modeling as the construction of mental models of phenomena through the use of 
conceptual models. Constructing a model—by identifying the salient features of the system or 
phenomenon under consideration and determining how those features, and the relationships 
among them, can be depicted or represented—is accompanied by using the model to illustrate a 
system, explain a system or phenomenon, or to make predictions about a phenomenon. This 
model construction leads to evaluating and revising models in light of findings so they will better 
achieve their intended purposes. 

With the development of technology, modeling often takes place in virtual worlds using 
computer simulations. Computer simulations are programs that contain a model of a system 
(natural or artificial, e.g., equipment) or a process. In simulation-based learning environments, 
the main task of the learner is to infer the characteristics of the model underlying the simulation. 
The learners’ basic actions are changing values of input variables and observing the resulting 
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changes in values of output variables. De Jong and van Joolingen (1998) found that computer 
simulations provide support in the following areas of learning: generating hypotheses, designing 
experiments, making predictions, and regulating learning processes. Computational thinking has 
been increasingly stressed in learning sciences literature. Papert (1980) argued that simulation- 
supported environments bring in such mindstorms in which students can formulate and test 
alternative hypotheses and reconcile the discrepancy between their ideas and the observations in 
a microworld. That is, the computers help students to discover the discrepancy by providing 
contexts for students to test out their original hypotheses and showing the consequences of 
actions based on their hypotheses. 

Scientific modeling, therefore, can be decomposed into several components: constructing, 
using, evaluating, and revising models (Schwarz et al., 2009). These components are further 
elaborated below: 

• Constructing models: One can use a hypothesis informed by principles or theories to 
construct models and apply appropriate principles, theories, or evidence. 

• Using models: One can use multiple models to explain and predict aspects of a group 
of related phenomena. 

• Evaluating models: One should consider the predictive or explanatory power of a 
model and determine whether a model makes accurate and reliable predictions, 
whether it accounts for all available evidence, and/or whether there is consistency 
with all available evidence. 

• Revising models: One should revise a model when predictive power or explanatory 
power is called into question or when new evidence is available that challenges 
predictive or explanatory power. 

Some researchers have collected evidence that modeling does support students’ ability to 
learn concepts (Penner, Giles, Lehrer, & Schauble, 1997; White, 1993) because modeling 
requires students to express and test ideas by collecting and evaluating evidence. This process of 
model testing and revising is believed to facilitate conceptual development. 

We expect that students may have some experience constructing models (particularly 
physical models) in classrooms; however, few students may have experience using, testing, or 
revising models. Therefore, when assessing students’ modeling skill, it is necessary to provide 
some examples of how scientists use, test, and revise their models. In addition, being proficient 
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in modeling includes self-conscious separation of a model and its referent, as well as the explicit 
consideration of measurement error, and consideration of alternative models (Lehrer & Schauble, 
2000). Therefore, when assessing students’ modeling skills, it is also important to prompt 
students to consider alternative models and limitation of models. 

Science Practice 4: Engaging in empirical investigations. Scientists often design and 
conduct investigations to develop and test theories that answer questions about how the world 
works. Students need opportunities to engage in empirical investigations to test explanatory 
theories of the world and gather data to support or refute their predictions. Engaging in empirical 
investigations is a multifaceted activity that includes a set of procedural and conceptual 
activities, such as asking questions; generating hypotheses; designing experiments; collecting 
and recording data; analyzing, representing, and interpreting results; coordinating theory and 
evidence; and fonnulating and revising theories or models (Duschl et ah, 2007). Below, we 
elaborate on a few of these subcomponents included in the practice of engaging in empirical 
investigations: 

• Asking questions is an important component of science literacy because an essential 
aspect of science is to develop explanations or models that address questions about 
phenomena in the natural world. We emphasize that the questions scientists attempt 
to account for occur in the natural world, not the human-made world (the latter is the 
domain of engineering or a technological field such as computer science). The type of 
question that a student can ask can indicate the level of sophistication in that student’s 
thinking (Yarden, Brill, & Falk, 2001). In classrooms, questions usually are initiated 
by teachers and rarely by the students. Therefore, students are not likely to ask 
questions. Even if they do ask questions, the nature of questions that most students 
ask reflect a desire to seek clarification or declarative knowledge (e.g., what are 
bacteria?), rather than a desire to understand underlying scientific mechanisms or 
procedural knowledge (e.g., how do seasons form?). Cuccio-Schirripa and Steiner 
(2000) suggested that “Questioning is one of the ... processing skills which is 
structurally embedded in ... critical thinking, creative thinking, and problem solving’ ’ 
(p. 210). Thus, questioning may contribute in important ways to the development of a 
scientific theory because it is one potential bridge between theoretical ideas and the 
empirical investigation that includes data collection, interpretation, and evaluation of 
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evidence needed to test or revise ideas (a key feature of critical thinking and problem 
solving). Accordingly, it is important to help students ask well-formulated questions 
that can be answered through empirical investigation. 

• Generating a hypothesis involves proposing a rational and tentative explanation of an 
observed phenomenon that has not been proved. A hypothesis is often produced to 
identify the relationship between a dependent variable and the independent variable. 
Many hypotheses are stated in the form of “if a particular independent variable is 
changed, there is also a change in a certain dependent variable.” The hypothesis 
developed is the goal of the designed experiment. Such practice helps students plan 
their data collection by considering variables and expected observation involved in 
their inquiry. 

• Designing experiments involves identifying causal variables and generating 
interpretable observations that will serve as evidence—evidence that can be related to 
a hypothesis about the proposed questions. One important aspect of design is to 
isolate variables so as to rule out competing hypotheses. A well-designed 
investigation often includes controlled variables that allow valid inferences and 
narrows the number of possible experiments to consider (Klahr, 2000). Another 
important aspect of designing investigations is selecting appropriate measures or tools 
to examine the phenomena of interest (e.g., determining if the total mass or the 
density of a material might be a better measure of a property to identify a material’s 
identity, or detennining if height or mass is a better measure of growth). Students 
may also need to understand certain aspects of measurement error and the need for 
accurate and precise measurements that might require appropriate or specific tools 
(e.g., a balance is a more accurate measure of the comparative masses of two different 
objects than relative heft in a person’s hand). There might also be a need to appreciate 
sample size (i.e., are there enough observations to make a generalizable inference) 
and the role of randomness in selecting a sample (i.e., the potential for bias because 
the sample is unlikely to be a true representation of the population). 

• Collecting and recording data is also an important component of engaging in 
empirical investigation because access to interpretable, useful, valid, and reliable data 
is essential to making effective evidence-based explanations or arguments. A datum is 


14 



an observation or measurement of a natural system or of a designed and constructed 
experimental situation recorded for subsequent analysis. Research has shown that 
students were unlikely to check if a current hypothesis is not consistent with 
experimental results (Dunbar & Klahr, 1988). The access to collected data may help 
students realize such gaps. Students must identify the best ways to record data 
accurately and reliably. 

• Analyzing and interpreting data often follows after data collection. This is when 
students make sense of data and connect the data as evidence to support their 
hypothesis. This process is often related to students’ skills of using mathematical and 
computational tools. The NRC framework states that mathematical tools (e.g., 
formulas, mathematical rules behind the simulational programming) enable students 
to express ideas in a precise form, and computational tools (e.g., simulations, 
animations) enable students to represent data and scientific phenomena visually. Both 
types of tools allow students to explore and identify patterns in data and their 
observations. It is also recognized that students’ ability to engage in mathematical 
thinking is related to their ability to identify patterns in data and represent data in 
appropriate ways. In the domain of science, mathematical thinking skill is also related 
to the ability to effectively design or select a measure for a given variable, 
particularly in simulations. For example, students need to apply their mathematical 
thinking skills to consider what value should be set for the variable of the amount of 
water and salt in a simulation that helps to understand the concept of density. 

Computational tools, such as computer simulations, provide cost-effective ways of 
doing experiments. Students can use simulations to observe scientific phenomena that 
cannot be observed easily in real time (e.g., volcano eruption, protein synthesis, and 
spread of disease). The enabled visualization of scientific phenomena and interactive 
opportunities provided by computer simulation has been associated with gains in 
conceptual understanding and enhances epistemological understanding about the 
nature of science as exploring and discovering knowledge through multiple trials of 
experimentation and evidence-based reasoning (Wilensky & Reisman, 2006; 

Zacharia, 2007). From the aspect of assessment, computer simulations also allow 
testing of both conceptual understanding and the ability to engage in the central 


15 



practices of science that are not well tested in traditional assessments (Quellmalz & 
Pellegrino, 2009). 

One major challenge with the practice of engaging in empirical investigation is that often 
there is little classroom instruction focused on designing and carrying out a high-quality 
investigation and analyzing and interpreting data collected from the investigation. Therefore, it is 
necessary to scaffold students as they attempt to complete assessment tasks that target students’ 
ability to engage in empirical investigations. Another challenge is that it takes time to engage 
students in a full cycle of empirical investigation, a particular problem for short assessments. 

One possible approach is to target pieces of the practice of engaging in empirical investigation, 
such as asking questions, designing an investigation, or analyzing data in one task. Alternatively, 
we may also use surrogate methods to measure individual practices (e.g., using multiple-choice 
items asking students to choose from a list of potential procedures). In addition, we might design 
extended tasks that will be completed across multiple class periods so that students have time to 
engage in the various aspects of empirical investigation. Furthermore, we may ask students to 
conduct an experiment in a simulation-based setting in which they can take less time than in a 
traditional real-time lab. 

Science Practice 5: Communicating and collaborating. Communicating and 
collaborating is an essential competency to achieve. Knowledge-building theorists have 
emphasized the social aspect of learning. Scardamalia and Bereiter (2006) summarized six themes 
that underlie a shift from treating students as individual learners to regarding them as members of 
a knowledge-building community: (a) knowledge advancement as a community rather than 
individual achievement; (b) knowledge advancement as idea improvement rather than as progress 
toward true or warranted belief; (c) knowledge of science (the ability to do science, e.g., knowing 
how to construct, evaluate, and revise a model of matter) in contrast to knowledge about science 
(declarative knowledge, e.g., knowing that atoms are subcomponents of molecules)', (d) discourse 
as collaborative problem solving rather than argumentation; (e) constructive use of authoritative 
information; and (f) understanding as an emergent process. All these themes recognize the 
fundamental role of communication and collaboration in creating a knowledge-constructing 
culture. Roschelle (1992) defined collaboration as a practice conducted by two or more people 
whose goal is to reach convergent understanding or to construct shared meanings through 
communication. We take the broader definition of collaborative learning as “a situation in which 
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two or more people learn or attempt to leam something together” (Dillenbourg, 1999, p. 1). In this 
definition, there are two important aspects of collaborative learning. First, it is about interactions 
between two or more people. As Dillenbourg (1999) argued, true collaboration includes both 
interactivity and negotiability. Interactivity refers to the impact of interactions on group members’ 
thinking and learning. Negotiability refers to the important role of negotiation among group 
members in order to reach convergent or shared understanding. In other words, instead of 
imposing one’s knowledge on other group members, new knowledge is built upon individual 
group members’ contributions and further modified through negotiation. Second, it is about 
learning together as a group with a shared learning goal. In this sense, collaborative learning is 
different from cooperative learning. In cooperative learning, students work in structured groups to 
complete a task by breaking apart the task into different pieces and assigning these different 
pieces to individuals within groups. However, in collaborative learning, students work together to 
achieve a common group goal. Individual group members contribute to the group task and each 
contribution is considered and discussed by all group members for agreement or disagreement. 

Productive collaborative learning requires both individuals’ independent skills and their 
interdependent skills. Essential independent skills include the capability of listening to and 
evaluating feedback to others’ opinions, openly considering modifying one’s own thinking based 
on the feedback from others, and assimilating and accommodating one’s own thinking with 
others’ thinking. Some important interdependent skills include communicating one’s own ideas 
to the audience, communicating to better understand other members’ ideas (e.g., paraphrasing the 
contributions of others and asking clarification questions), and efforts to help achieve group 
learning goals. 

Communication and collaboration have been emphasized in the skills framework for 21 st 
century learning (Partnership for 21st Century Skills, 2007) because it is important for scientists, 
engineers, and technology professionals (as well as citizens working in the global workplace) to 
be able to articulate and listen to thoughts and ideas effectively and to collaborate with diverse 
teams. Effective collaboration involves sharing responsibility for work and valuing the individual 
contributions made by other team members. The distributed nature of cognition suggests that 
learning requires communication among people (Pea, 1993). Collaborative communication 
creates an awareness of the need for knowledge revision and encourages deep processing; 
therefore, it is a powerful tool to facilitate conceptual change (Liu & Hmelo-Silver, 2010; 
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Roschelle, 1992). Intersubjective meaning making (i.e., negotiating between participants to reach 
convergent understanding or identify and resolve disagreements) in collaborative communication 
helps create joint interpretations through phases of negotiation focused on shared infonnation 
(Suthers, 2006). As Liu and Hmelo-Silver (2010) summarized, peer collaborative interactions 
promote learning by arousing an awareness of the need to revise knowledge, initiate knowledge 
reconstruction, and encourage deep processing. Peer communication provides opportunities to 
recognize such needs. 

There is a lot of research about collaborative learning with a focus on assessing students’ 
learning outcomes in collaborative settings or evaluating the impact of specific collaborative 
learning environments or tools. However, few studies focus on assessing students’ practice of 
collaboration and communication. In general, it is quite challenging to address the issue of 
assessing collaboration and communication because collaborative learning includes complex 
interactions among multiple participants and can involve a diversity of learning goals. It is 
important for us to understand these challenges and think about possible solutions. One major 
challenge to assessing collaboration and communication is that there are different kinds of 
collaborative activities. For example, collaboration may take place when groups of students 
investigate a scientific phenomenon, when students engage in game-like tasks or simulations, or 
when students participate in discussion groups with a goal to construct knowledge bases. All 
these learning activities can be assessed at individual and group levels (D. W. Johnson & 
Johnson, 1992). Another challenge with assessing this practice is that the data collected during 
students’ collaboration and communication are quite different from traditional assessment data. 
Therefore, different analysis methodologies need to be applied. 

Dimension 3: The Crosscutting Concepts 

The crosscutting concepts provide one way of linking across the domains in science. 

They include concepts that bridge disciplinary boundaries and/or an organizational framework 
for connecting knowledge across domains into a coherent view of the world. We identified three 
crosscutting concepts to be included in our competency model: systems thinking, cause and 
effect, and flow and conservation of matter and energy. We identified these crosscutting 
concepts from the work by Duschl et al. (2007). We focus on only three crosscutting concepts 
initially because we see them as having particular relevance to the content focus of our current 
CBAL science work (e.g., matter in the physical sciences and, later, structures and processes and 
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ecology in the life sciences). One crosscutting concept in particular, systems thinking, has been 
given some attention by researchers in the learning sciences. 

Crosscutting Concept 1: Systems thinking. To foster science literacy, it is essential to 
engage learners in systems thinking (Sabelli, 2006) because it facilitates integrating knowledge 
across science domains and provides a unifying concept for learners to make sense of the natural 
world and relate microscopic components underlying a particular phenomenon to macroscopic 
observations detected in our visible world. For example, it is often challenging for students to 
understand how the interactions among particles at the microscopic level (or, as some would say, 
the nanoscopic level) are related to the properties of matter at the macroscopic level. Systems 
thinking is required to develop such an understanding. Unfortunately, most of our current K-12 
science instructional materials do not intentionally target systems ideas. 

Systems thinking is derived from the simulation modeling field of system dynamics 
(Forrester, 1961; Stennan, 2000). A system is “an entity that maintains its existence and 
functions as a whole through the interactions of its parts” (Assaraf & Orion, 2005, p. 519). Many 
scientific phenomena around us are examples of systems, such as ecosystems, moon phase 
formation, and energy transfer. How one perceives the phenomena often is related to systems 
thinking skills. Systems thinking tends to involve the analysis of scientific phenomena and 
problems in wider contexts, considers multiple and nonlinear cause-and-effect relationships, and 
understands change over time (Hogan, 2000; NRC, 1996). Systems thinking is a skill that is 
related to both the nature of science and nature itself, but it is often neglected in the design of 
learning and assessment environments (Golan & Reiser, 2002; Wilensky & Resnick, 1999). 

Crosscutting Concept 2: Cause and effect. Most scientific investigations are to answer 
questions that explore cause-and-effect relationships, such as the following: Why did that 
happen? How did that happen? What mechanisms caused that to happen? Therefore, the 
application of science is dependent on understanding the cause-and-effect relationships between 
events and elucidating the mechanism for mediating the relationship between two components of 
investigation in a system or in a particular phenomenon. Research shows that it is human beings’ 
innate nature to explore the causal relationships between events (Ofer & Durban, 1999). Students 
at a very young age tend to develop some causal explanations to make meaning of the world, 
although their explanations often concern the superficial features of events. It may be simple to 
identify patterns in events, in which events occur together, and such patterns might represent real 
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cause and effect relationships, but proving such relationships or elucidating the details of the 
relationship requires more investigation. The fonnal science education aims to help students 
appreciate standard scientific theories that explain the causal mechanisms under study and 
develop arguments based on evidence when attributing an observed phenomenon to a specific 
cause. 

Crosscutting Concept 3: Flow and conservation of matter and energy. According to 
the law of conservation of matter, matter can be transfonned from one form to another, but it 
cannot be created or destroyed. Similarly, according to the first law of thermodynamics, energy 
can be transfonned from one form to another, but it cannot be created or destroyed. The concept 
of flow and conservation of matter and energy can be informative in understanding systems in all 
science domains—such as life science, physical science, and earth science—thus, tracking the 
transfers of matter and energy has important implications for developing interdisciplinary 
understanding. Conservation is a particularly important principle to apply to all phenomena 
involving matter and energy, yet students often fail to apply or use it appropriately. For example, 
when it comes to matter, students often fail to recognize that matter is conserved during a 
chemical reaction or a phase change that results in the production of a gas. In such cases, many 
students believe that what they cannot see simply disappears from existence in the universe, 
when, in fact, the opposite is true. Students should be able to track or account for energy and 
matter in terms of inputs, outputs, flows, or transfers within a system or process (NRC, 2012). 
With the proper understanding and application of the flow of matter and energy, students in 
middle school should, for example, be able to recognize that a plant or animal cannot grow 
properly without a sufficient input of matter and energy, and students should be able to account 
for where the matter is coming from and where it ends up as it is transformed inside of 
organisms. 

Dimension 4: Epistemic Beliefs/Metacognition 

Epistemic beliefs refer to beliefs about knowledge (including its structure and certainty) 
and knowing (including sources and justification of knowledge; Buehl & Alexander, 2001; Duell 
& Schommer-Aikins, 2001; Hofer, 2000; Hofer & Pintrich, 1997). Increasingly, educational 
researchers have become interested in how epistemic beliefs about knowledge and knowing are a 
part of the process of learning and instruction and how these beliefs affect or mediate the 
knowledge acquisition and knowledge construction processes. There need to be changes in 
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students’ and teachers’ epistemic cognition and views about the nature of science (Duit & 
Treagust, 2003) in order to construct new knowledge effectively in science. Most students do not 
seem to have an epistemology of science that is consistent with current inquiry-based approaches 
to learning science, and few students see science as a process of building and testing models and 
theories. Instead, science is seen as a steady accumulation of facts about the world as piecemeal 
infonnation that is unconnected to everyday experience and is to be accepted because of the 
authority of the teacher or text (Carey & Smith, 1993; Driver, Leach, Millar, & Scott, 1996; 
Hammer, 1994; Linn & Songer, 1993; Liu & Hmelo-Silver, 2009; Smith, Maclin, Houghton, & 
Hennessey, 2000). Research has shown that students’ epistemic beliefs about the structure of 
knowledge, as well as the construction and stability of scientific knowledge, predict better 
learning gains and that only students holding constructivist epistemic beliefs achieve a deep 
conceptual understanding of scientific knowledge, such as Newtonian dynamics (Stathopoulou & 
Vosniadou, 2006). 

To improve students’ epistemic understanding of the nature of science, it is essential to 
promote teachers’ understanding in this area (Yoon, Liu, & Goh, 2009). Research shows that 
teachers who believe that knowledge is derived from experts and that one’s learning ability is 
innate are more likely to engage in traditional, teacher-centered instructional practices. In 
contrast, teachers who believe that knowledge is constructed from one’s experiences and 
judgment, that knowledge is tentative and changing, and that one’s ability can be changed tend to 
conduct progressive instructional practices that are student-centered (Chan & Elliott, 2004). 
Therefore, educational researchers and practitioners call for approaches to shifting teachers’ and 
students’ epistemic beliefs from an absolutist (knowledge is objective, located in the external 
world, and certain) to a constructivist (knowledge is constructed and uncertain) view. We 
propose an approach to using formative assessment that includes many opportunities to engage 
students in epistemic practices, learning experiences that focus on the development of personal 
epistemologies through engaging learners in learning activities of doing science, to help foster 
the shift of epistemic beliefs. The assumption is that if students are taught science in the context 
of inquiry, they will know what they know, how they know it, and why they believe it (Duschl, 
2003). This approach requires that instruction be continuously modified while learning is taking 
place. This continuous modification is where formative assessment comes into effective 
instruction—when appropriately designed and implemented formative assessment involves 
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gathering, interpreting, and acting on information about students’ learning so that it may be 
improved (Bell & Cowie, 2001; Duschl, 2003) and can support learning (Black & Wiliam, 

1998). 

Epistemic beliefs are often related to one’s metacognitive knowledge (metacognition). 
Metacognition is often referred to as thinking about thinking and can be used to help students 
learn how to learn. Specifically, metacognition refers to students’ automatic awareness of their 
own knowledge and their ability to understand, control, and manipulate their own cognitive 
processes (Flavell, 1979). Metacognition consists of several essential elements: planning 
(developing a plan of action), monitoring (maintaining the plan), and evaluating (evaluating the 
progress). Below, we elaborate on these elements: 

• Planning'. During the planning process, students need to have an understanding of 
what prior knowledge they have and whether it is useful for a particular task. They 
also need to know what other knowledge is needed to complete the task. 

• Monitoring: During the monitoring process, students attempt to keep themselves on 
the right track of problem solving and iterate back and forth between the planning 
process, the use of knowledge, and the learning strategies needed to achieve the 
targeted learning goals. 

• Evaluating: During the evaluating process, students assess whether they have reached 
their goals; assess what gaps in their knowledge, skills, or abilities still need to be 
filled in; and, most importantly, recognize how they obtained the necessary 
knowledge for problem solving. 

Metacognition has been interchangeably used with the tenn self-regulation, which 
emphasizes students’ ability to adjust their learning processes in response to their perception of 
feedback regarding their current status of learning. 

Engaging students in metacognitive processes guides students’ thinking as they work 
through a problem and make decisions. Davidson and Sternberg (1998) have argued that 
metacognitive knowledge allows the problem solver to better encode and represent the givens in 
a problem context (i.e., what information is provided in the problem), break down the problem 
into smaller questions that are relevant, and therefore derive a better solution. 

It is particularly challenging to assess students’ metacognitive processes, as those 
processes are often hidden in thinking. Traditionally, students’ metacognition is assessed through 


22 



Likert-type self-report surveys (Pintrich, Smith, Garcia, & McKeachie, 1993; Weinstein, 
Zimmennann, & Palmer, 1988). Other research has focused on providing metacognitive 
scaffolds to make evidence of students’ thinking visible by asking students to think aloud 
verbally or to write down their thinking (e.g., Azevedo & Hadwin, 2005; Saye & Brush, 2001). 
To this end, some virtual learning environments in science, such as ThinkerTool and WISE 
(Slotta, 2004; White, 1993), build in functions to log students’ pathways to problem solving so 
that students can visualize their thinking process during problem solving. Although there have 
been some advances in the measurement of metacognition, more work establishing the reliability 
and validity of the available measures is needed. 

Dimension 5: Motivation 

According to Brophy (1988), motivation to learn is “a student’s tendency to find 
academic activities meaningful and worthwhile and to try to derive the intended benefits from 
them” (pp. 205-206). In our competency model, we define motivation as the values, interest, and 
student engagement aroused by an assessment prototype. Poor motivation in assessment 
prototypes leads to low achievement on them. Therefore, in assessment development, special 
attention needs to be paid to improving student motivation. For example, designing an authentic 
and meaningful scenario context is essential to engaging students in applying scientific 
knowledge to explain scientific events related to their daily life. Another strategy to increase 
students’ motivation in taking an assessment is to provide immediate feedback. If students have 
experienced success in earlier perfonnance, they are more likely to feel positively toward a new 
task. It is important to provide feedback that can help students realize what knowledge they have 
mastered and what knowledge they need to master or construct to be able to complete the task. In 
other words, the feedback should be task-oriented rather than person-oriented. In addition, new 
interactive technologies provide opportunities to actively involve students in problem solving by 
doing science (e.g., designing and conducting experiments). Such interactive features in science 
assessment can also help to maintain students’ motivation during the task. 

Learning Progressions 

Learning progressions (LPs) in science have been defined as “empirically grounded and 
testable hypotheses about how students’ understanding of, and ability to use, core scientific 
concepts, explanations, and related scientific practices grow and become more sophisticated over 
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time, with appropriate instruction” (Corcoran, Mosher, & Rogat, 2009, p. 20). Essentially, an LP 
is a road map that shows how students’ qualitative understanding of particular ideas or practices 
is likely to change over time with appropriate instruction. Therefore, LPs can serve as a guide for 
educators as they design instruction and monitor students’ progress toward the targeted levels of 
understanding and ability. According to an expert panel review of the work in this field, a 
learning progression must include the following elements: 

1. Target perfonnances or learning goals that are the end points of a learning 
progression and are defined by societal expectations, analysis of the discipline, and/or 
requirements for entry into the next level of education. 

2. Progress variables, which are the dimensions of understanding, application, and 
practice, that are being developed and tracked over time. These may be big ideas that 
constitute core concepts in the discipline or critical aspects of practices central to 
scientific work. 

3. Levels of achievement that are intennediate steps in the developmental pathway(s) 
traced by a learning progression. These levels may reflect levels of integration or 
common stages that characterize the development of student thinking. There may be 
intennediate steps that are noncanonical but are stepping stones to canonical ideas. 

4. Learning perfonnances that are the kinds of tasks students at a particular level of 
achievement should be capable of performing. They provide specifications for the 
development of assessments and activities that identify the scientific ideas and 
practices by which students will demonstrate their proficiency in science. 

5. Assessments are the specific measures used to track student development along the 
hypothesized progression. Assessments are integral to the development, validation, 
and use of LPs. 

6. They can be validated and empirically tested (Corcoran et ah, 2009). 

Based on this consensus and the goals of CBAL, we presume that learning progressions 
should be based on learning theories of how students gain particular core knowledge and skills 
(or practices) within a domain, as well as on the developmental theories of how students develop 
particular cognitive skills over time. In sum, LPs are working hypotheses of how students’ ideas 
and abilities change over time as the effect of instruction, learning, and cognitive development. 

In addition, LPs are continually revised based on the collection of new evidence from the 
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students. The empirically based revisions enable LPs to provide more accurate and general 
descriptions of sequentially different patterns, or levels, in thinking and ability, as students make 
progress toward the upper level of understanding defined in standards. In the field of assessment, 
LPs provide a framework to guide the design of tasks that are sensitive to progress toward a 
desired level of proficiency. In particular, LPs can help design assessments that provide a range 
of infonnation about students’ intennediate knowledge, rather than evaluate students’ ability and 
knowledge as an oversimplified all-or-nothing category of response. Additionally, LPs can guide 
the development of scoring rubrics and suggest ways to elicit evidence of student thinking. 
Furthennore, LPs can also provide teachers with an opportunity to think critically about what 
students know, and can do, and realize where students might be coming from and where they 
need to go; therefore, LPs can serve an important role in supporting the formative assessment 
process (Furtak, 2012). 

A review of the work on LPs in science shows that only a few dozen hypothetical LPs 
have been developed (Corcoran et ah, 2009). Even fewer LPs have been developed that span all 
grades in school (K-12). Moreover, only a small proportion of hypothetical LPs have been tested 
and validated. There certainly are not enough hypothetical LPs to cover all the current NGSS 
standards. The biggest gaps are in the earth and planetary sciences, where there are very few 
hypothetical LPs. There are more hypothetical LPs in the life sciences (in particular, ecology and 
evolution) and the physical sciences (in particular, the structure of matter). However, there are 
some very fundamental topics for which we do not have LPs (in particular, energy and cell 
theory). Perhaps the most well validated LPs that extend to middle school are for the flow of 
matter and energy in an ecosystem, that is, the carbon cycle (Mohan, Chen, & Anderson, 2009), 
and for food webs and food chains (Gotwals & Songer, 2006; Songer, Kelcey, & Gotwals, 2009). 
In the physical sciences, there is a fair amount of research on students’ alternative conceptions of 
matter and how students’ conceptualization of matter changes across elementary and middle 
school—enough research, in fact, to inform a provision K-8 LP for matter (Smith et ah, 2006). 
Current work at the elementary and middle school levels is attempting to validate parts of this LP 
(S. Y. Stevens, Delgado, & Krajcik, 2010; Wiser, Smith, & Doubler, 2012). Unfortunately, while 
ideas such as matter are believed to be critical to developing conceptual knowledge in other 
disciplines of science, such as the earth sciences, very little work has been done to examine the 
interactions between a matter LP and an LP for any other core idea. In fact, there are no studies 
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looking at the interactions between LPs for any two core ideas in science; hence, this is also a 
major gap in the literature (Corcoran et al., 2009). Furthermore, little work has been done to 
examine the interactions between specific LPs in science and those in very different domains, 
such as mathematics or English language arts. 

Many researchers argue that, parallel with developing conceptual knowledge, students’ 
ability to engage in and understand particular science practices, such as constructing models, 
constructing evidence-based explanations, designing investigations, or constructing arguments, 
also grows in sophistication over time. A few researchers have developed LPs that foreground 
and focus on the development of science practices, such as the work by Schwarz and colleagues 
on modeling (Schwarz et al., 2009) or on explanations (Berland & McNeill, 2010), but, again, 
there are only a few learning progressions focused on particular science practices. However, if 
meaningful competency models are to be developed that are consistent with current policy 
recommending students engage in these science practices (NRC, 2012), then progressions 
covering such practices should be included. The term progression, in this case, would refer to 
students’ increasing capacity to engage productively in the practice (e.g., developing more 
sophisticated models), as well as students’ increasing ability to reflect on the nature of the 
practice (e.g., developing more sophisticated epistemological understandings of models). Some 
researchers have proposed progressions that intimately connect the development of content 
knowledge and the development of students’ ability to engage in science practice, in particular, 
the work by Lehrer and Schauble (2012) on data modeling and biodiversity; however, this work 
is limited to the early and middle elementary grades. 

For this early stage of the project, we have focused on one core idea (matter, including 
the structure and properties of matter, and changes and conservation of matter). We have focused 
on matter because it is one of the most foundational core ideas in the natural sciences, and there 
is considerable research to draw from. 

Based on reviewing existing literature, we drafted a hypothesized LP for the core concept 
of matter and its interactions and a hypothesized LP for the science practice of constructing 
scientific explanations. Based on external feedback from advisory committees, we made multiple 
revisions to the hypothesized progressions. We chose to keep the progressions for content and 
practice separate, given that it might be difficult to disentangle the constructs if they were present 
in a fused progression. 
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Tables 1 and 2 present the LPs for two big ideas that make up the core idea of matter and 
its interactions: (a) properties and structure of matter and (b) change and conservation of matter. 
The development of both LPs is based on theoretical and empirical scholarly work, although they 
still need to be tested and validated, which is planned in our future work when we pilot and 
implement the tasks. The theoretical work for the matter LPs is largely based on work by Smith 
et al. (2006) and Rogat et al. (2011). The Smith et al. work theorized a K-8 learning progression 
for the atomic-molecule model of matter. This work identified several big ideas as key aspects of 
student thinking that our progression addresses, such as material identity, properties of matter 
relevant to structure of matter, and conservation and transfonnation of matter. The more recent 
work by Rogat et al. also had important impact on developing our progression. This work 
theorized K-12 progressions for properties and structure of matter, change in matter, and more 
current understanding of how students come to understand matter. Furthermore, this work is well 
aligned with A Framework for K-12 Science Education (NRC, 2012). 

The development of the LPs was also based on empirical work (P. Johnson, 1998, 2000, 
2002; Krajcik, McNeill, & Reiser, 2008; Krnel, Watson, & Glazar, 1998; Liu & Lesniak, 2005; 
Merritt, 2010; Nakhleh & Samarapungavan, 1999; Papageorgiou & Johnson, 2005; Stavy, 1990, 
1991; S. Y. Stevens et al., 2010; Wiser & Smith, 2008; Wiser et al., 2012). These different 
empirical pieces range from cross-sectional and short-term longitudinal studies examining 
students’ conceptions of matter through interviews and drawings, to curriculum interventions 
that attempt to support and examine students’ conceptions of matter through drawings and/or 
explanations. 

In sum, our LPs for matter and its interactions show a progression from early 
macroscopic notions of matter that include no conception of the nanoscopic particles of matter, 
or their structure or behaviors (e.g., Levels 1-2 in the structure and properties LP as well as in 
the change and conservation LP), to a variety of intennediate conceptions of matter that include 
increasingly more sophisticated notions of the nanoscopic particles of matter and their structure 
and behaviors (e.g., Levels 3-4 in the structure and properties LP and Levels 3-5 in the change 
and conservation LP). Finally, at the top level of the progression are the most sophisticated 
models of matter expected from middle school science students as articulated in the NGSS 
(Level 5 in the structure and properties LP and Level 6 in change and conservation LP). At these 
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top levels, students are capable of explaining, predicting, or modeling the properties and 
behavior of a variety of natural phenomena involving matter. 


Table 1 


The Learning Progression for Properties and Structure of Matter 


Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 1—Macroscopic compositional model 


Material identity and characteristic 
properties of matter 

• Has a macroscopic notion of material 
identity: realizes some materials can 
change form and still remain the same 
material (e.g., by melting) or realizes 
that an object broken into smaller 
pieces that still have perceptual 
properties of the original material (e.g., 
a piece of paper cut into small pieces) 
is still the same material identity. 

Pieces of matter 

• Recognizes that objects can be broken 
down into smaller pieces that have 
weight and volume. 

Properties of matter in different states 

• Can distinguish general properties of 
solids and liquids (e.g., liquids are 
running but solids are not). 

Key words: observable macroscopic 
properties of liquids and solids, 
historical thinking 


Material identity and characteristic 
properties of matter 
Conceives of matter or materials as 
what you can touch and feel. 

Tends to classify different types of 
materials based on historical thinking 
(i.e., tend to think about where 
materials came from to classify 
them— e.g., plastic is man-made and 
therefore is a material, whereas wood 
is from nature and therefore is not a 
material). 

May not distinguish extrinsic from 
intrinsic properties (e.g., mass and 
volume may be viewed as intrinsic 
properties). 

Pieces of matter 

Does not recognize that objects can be 
broken down into very much smaller 
pieces (e.g., the size of a grain of sand) 
that have weight and volume. 

Properties of matter in different states 

Sees solids and liquids as 
fundamentally different. 


Help students understand that if one 
divides an object into very small 
pieces that appear not to have the 
perceptual properties of the original 
object (e.g., the size of sand grains 
from grinding a rock), those tiny 
pieces still have the same physical 
intrinsic properties of the original 
object (color, texture, hardness, etc.) 
and can be seen by using sensitive 
instruments such as magnifying 
glasses. Also help students realize 
these tiny pieces also have weight and 
volume by using sensitive instruments 
like scales or graduated cylinders. 

Help students understand that liquids 
and solids of the same material are 
fundamentally similar (e.g., solid and 
liquid butter or solid and liquid water) 
by visualizing the state change from 
liquid to solid and vice versa and 
exploring the intrinsic properties of the 
material; students should work with 
evidence that intrinsic properties of 
matter are constant, even though the 
extrinsic properties like mass and 
volume can change. 

Help students realize that solids and 
liquids are fundamentally similar in 
that they both have mass and volume; 
bridging analogies can also be helpful, 
such as exploring aggregates to show 
how volume and mass are extrinsic 
properties of both solids and liquids 
and both can be divided into smaller 
units that, if added up, can equal the 
sum of the original object or sample. 


Level 2—Microscopic compositional model 


Material identity and characteristic 
properties of matter 

• Has a microscopic compositional 
notion of material identity: realizes 
that an object broken into smaller 
pieces that no longer have the 
perceptual properties of the original 
object is still the same material identity 
(e.g., wood vs. sawdust). 

Pieces of matter 

• Recognizes that an object can be 


Material identity arid characteristic 
properties of matter 

• Still may conceive of matter or 
materials as what you can touch and 
feel. 

• May sometimes classify different types 
of materials based on historical 
thinking. 

• Does not recognize that different 
properties of matter are determined by 
the arrangement and motion of 


• Help students understand that intrinsic 
properties are unique to matter and are 
related to the nanoscopic particles that 
compose matter by using physical 
and/or computer modeling tools. 

• Help students understand that gas is 
another form of matter by constructing 
models and using representations to 
explore and make sense of the state 
change from liquid to gas and vice 
versa. 
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Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 1—Macroscopic compositional model 

broken down into very small pieces 
(e.g., grains of sand) that have weight 
and volume. 

particles making up the matter. 

• May ignore some intrinsic properties of 
matter (such as odor). 


Properties of matter in different states 

Pieces of matter 


• Recognizes that all solids and liquids 
are fundamentally similar in that they 
all have mass and volume and 

• Does not have a nanoscopic notion of 
material identity, but has a 
microscopic notion of material 


recognizes that matter may come in 
solid or liquid forms. 

Key words: material kinds of solids and 
liquids, mass and volume, historical 
thinking 

identity. 

Properties of matter in different states 
• Does not recognize gases as a form of 
matter or materials. 


Level 3—Developing particle model 


Material identity and characteristic 
properties of matter 

• Has a developing nanoscopic notion of 
material identity: sometimes 
recognizes that different materials are 
made of different particles (e.g., wax is 
made of wax particles and water is 
made of water particles). 

Pieces of matter 

• Conceives of matter as made of 
particles that have mass and volume. 

• Sometimes thinks of empty space 
between particles for some materials. 

• Sometimes recognizes that particles of 
matter move. 

Properties of matter in different states 

• Recognizes that all three states of 
matter (i.e., solid, liquid, gas) are 
fundamentally similar in that they all 
occupy space and have mass and 
volume. 

Key words: particles with macro 
properties _ 


Material identity and characteristic 
properties of matter 

• Sometimes may not recognize some 
materials or substance as being 
composed of particles. 

•May associate macroscopic properties 
(e.g., hardness or temperature) with 
individual particles. 

• Does not recognize the properties of 
materials are determined by the 
collection of the particles’ moving and 
interacting with each other. 

Pieces of matter 

• May not recognize that there is empty 
space between particles in all 
conditions, although having a 
nanoscopic notion of material identity. 

• May not recognize particles move for 
all substances and all states of matter. 
Properties of matter in different states 

• May not effectively incorporate ideas 
about motion and spacing of particles 
to matter that exists in different states. 


• Help students understand that different 
properties of matter are determined by 
the arrangement and motion of 
particles making up the matter by 
using computer simulations. Provide 
investigative opportunities for students 
to explore the relations between 
properties of matter and the 
arrangement and/or motion of 
particles. 

• Using computer-based visualization 
tools, help students recognize that 
there is empty space between particles 
in all three states, but the space 
between particles in the three states is 
different. 


Level 4 Particle model 


Material identity and characteristic 
properties of matter 

• Has an established nanoscopic notion 
of material identity: consistently 
recognizes different materials are 
made of particles. 

• Recognizes the properties of a material 
are determined by the collection of the 
particles’ moving and interacting with 
each other. 

Pieces of matter 

• Consistently conceives of matter as 
made of particles that have mass and 
volume. 

• Consistently thinks there is empty 
space between the particles. 

• Consistently recognizes that particles 
of matter move. (Note: students are not 


Material identity and characteristic 
properties of matter 
■ May not recognize that different 
properties of matter are determined by 
the type and arrangement of atoms. 

Pieces of matter 

■May not consistently recognize that 
different materials are made of specific 
atoms, or combinations of atoms 
forming molecules, although may have 
a general particle model. 


Help students construct an atomic- 
molecular model that different 
particles are made of specific atoms or 
combinations of atoms forming 
molecules. Using physical or computer 
models to help them understand the 
difference between molecules and 
atoms. 

Provide investigative opportunities for 
students to construct arguments about 
the behaviors of matter undergoing 
chemical change by using an atomic- 
molecular model to evaluate which 
argument better explains and predicts 
chemical change. 
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Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 1—Macroscopic compositional model 


expected to think about absolute zero.) 


• Recognizes that temperature is a 
product of the average kinetic energies 
of the particles of the substance. 

Properties of matter in different states 

• Recognizes the macroscopic properties 
of solids, liquids, and gases are a result 
of the spacing and speed of nanoscopic 
particles. 


Key words: particles with mass and 
volume, properties as a result of a 
collection of particles, interactions 
between particles, consistent 
application of nanoscopic level of 
material identity _ 


Material identity and characteristic 
properties of matter 

• Has a refined nanoscopic notion of 
material identity: recognizes that 
different materials are made of specific 
atoms or combinations of atoms 
forming molecules and that the 
arrangement of the atoms and 
molecules determines the properties of 
the material. 

Pieces of matter 

• Conceives of matter as made of atoms, 
or molecules composed of atoms, and 
that the atoms have mass and volume. 

• Consistently thinks there is empty 
space between molecules. 

Properties of matter in different states 

• Recognizes different properties are 
determined by the arrangement and 
motion of particles making up the 
substance. 

• May include notion of attractive forces 
between particles to explain some 
properties of matter. 

Key words: particles with mass and 
volume, properties as a result of a 
collection of particles, interactions 
between particles, consistent 
application of nanoscopic level of 
material identity 


Level 5—Atom-molecular model 

Material identity and characteristic 
properties of matter 

• Does not recognize that atoms may be 
composed of smaller particles, such as 
electrons, neutrons, protons. (Note: 
this notion is high school content and 
would not be expected for middle 
school.) 

• May not include notion of attractive 
forces between atoms and/or 
molecules. 


N/A 
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Table 2 


The Learning Progression for Change and Conservation of Matter 


Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 1—Macroscopic compositional model 

Reshaping matter—a precursor 
to physical changes 
• Realize that cutting an object into 
pieces that still have perceptual 
properties of the original material (e.g., 
a piece of paper cut into small pieces) 
or reshaping or reordering the object 
does not change the amount and the 
material type of the object. (Note: we 
emphasize a general notion of amount 
of stuff not quite conservation of mass 
here.) 

Reshaping matter 

• May not realize micro level small 
pieces (e.g., sand grains or sawdust) 
still have mass and volume. 

Conservation of material and 
identity and mass 

• May not realize that the material type 
does not change during melting 
process of all solid objects. 

• Help students understand conservation 
of amount of stuff when grinding 
things into micro level pieces by 
engaging them in investigations that 
allow them to measure and record the 
mass during state change. 

• Help students understand conservation 
of material identity across all melting 
and freezing events. 

Conservation of material and 
identity and mass 



• Realize that some solid materials can 
melt, and the substances in the liquid 
form are still the same substance. 

Key words: cutting, reshaping, melting. 



Level 2—Microscopic compositional model 

Reshaping matter—a precursor 
to physical changes 

• Realize that cutting and grinding into 
micro level pieces that do not have 
perceptual properties of the original 
object does not change the amount and 
the material type of the object. (Note: 
we emphasize a general notion of 
amount of stuff not quite conservation 
of mass here.) 

Conservation of material and 
identity and mass 

• Understand that all solid materials can 
melt, and substance in liquid form is 
still the same substance. 

Key words: grinding, melting 

Conservation of material and 
identity and mass 

• May be unable to effectively predict 
conservation of mass during familiar 
physical changes (e.g., melting) and 
does not incorporate particle in models 
of physical change. 

• Help students understand the 
conservation of mass during physical 
changes (e.g., melting) by engaging 
them in investigations that allow them 
to measure and record the mass during 
state change and to analyze data from 
these investigations. 

• Help students predict conservation of 
mass during physical changes by 
engaging them in investigations that 
allow them to measure and record the 
mass during state change. 

• Help students develop a particle model 
for state change by engaging them in 
more opportunities to construct and 
revise models. Can also use multiple 
representations of physical change at 
the particle level to support student 
understanding. 

Level 3—Beginning particle model 

Reshaping matter — a precursor 
to physical changes 

• Realize that mass and volume are 
conserved when reshaping, cutting, 
and grinding (e.g., when dividing 
matter into very small pieces). 

Conservation of material and 
identity and mass 

• Have the notion that type and number 
of particles (not distinguishing atoms 
from molecules) are conserved across 
some physical changes (e.g., melting, 
freezing). 

Conservation of material and 
identity and mass 

• May be unable to predict effectively 
conservation of mass during all 
physical changes. 

• Help students understand the 
conservation of mass during physical 
changes (e.g., boiling and evaporation) 
by engaging them in investigations 
that allow them to measure and record 
the mass during state change and to 
analyze data from these investigations. 
May use computer simulations to trace 
particles. 

• Help students apply a particle model 
across more cases of physical change 
by engaging them in more 
opportunities to construct and revise 
models. 
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Achievement 


• Therefore, mass is conserved across 
some physical changes. 


Gap/challenge 


Instructional experience to support 
progression 


Key words: melting, freezing 


Level 4—Unidirectional particle model 


Conservation of material and 
identity and mass 


• Have the notion that type and number 
of particles are conserved across many 
physical changes consistently (e.g., 
dissolving, boiling, evaporation) but 
unidirectional. 

• Therefore, mass is conserved across 
many physical changes in some 
directions. 

Key words: dissolving, boiling, 
evaporation 


Conservation of material and identity 
and mass 

May be unable to apply conservation of 
matter to all physical changes 
bidirectional (e.g., matter is conserved 
in both evaporation and condensation). 


Help students understand that the 
conservation of mass is bidirectional in 
evaporation and condensation by 
engaging them in comparing open 
versus covered cups experiments. May 
use computer simulations to represent 
particle models during evaporation and 
condensation processes. 

Help students apply a particle model 
directionally across all physical 
changes by engaging them in more 
opportunities to construct and revise 
models. 


Conservation of material and 
identity and mass 

• Have the notion that particles are 
conserved across all physical changes 
bidirectional (e.g., both evaporation 
and condensation). 

• Therefore, mass is conserved across all 
physical changes in all directions. 

Key words: physical change; 
evaporation; condensation. 


Level 5—Bidirectional particle model 

Conservation of material and identity 
and mass 

• May be unable to apply conservation of 
matter to chemical changes. 

Energy during change 

• May not recognize that some chemical 
reactions release energy and some 
absorb thermal energy. 


Level 6—Bidirectional atomic-molecular model 


Help students understand that the 
conservation of mass applies to 
chemical changes. May use computer 
simulations to predict and test what 
chemical components are present 
before and after certain chemical 
reactions. Can also use multiple 
representations of chemical changes to 
support student understanding. 

Help students apply a particle model 
across chemical changes by engaging 
them in more opportunities to 
construct and revise models. 

Help students understand that energy is 
included in chemical change. May ask 
students to measure the change in 
temperature after a chemical reaction 
and then ask them to construct an 
explanation by using computer 
simulations that explore micro level 
phenomena. _ 


Conservation of material and identity 
and mass 

• Have the descriptive notion that the 
number and type of atoms are 
conserved during chemical change. 

• Can apply conservation of mass across 
all physical and chemical changes in 
all directions. 

Key words: physical change, 
evaporation, condensation _ 


Energy during change 

• May not include mechanistic accounts 
(e.g., attractive forces within and 
between molecule) for physical and 
chemical changes. 

Energy during change 

• Recognize that some chemical 
reactions release energy and some 
absorb thermal energy. 


N/A 
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Table 3 presents the LP for constructing scientific explanations. The development of this 
LP is largely based on an empirical work focused on an LP for evidence-based explanations that 
account for biodiversity (Songer et al., 2009). Although Songer et al. stressed the importance of 
integrating content and inquiry reasoning progressions as parallel templates that together 
constitute a learning progression for a focal topic, we intentionally separate the content and 
science practice progressions in our current work because there is no convincing empirical 
evidence that supports the parallel progression of these two dimensions. The development of our 
LP for constructing scientific explanations is also influenced by the theoretical work of McNeill 
and Krajcik (2008) and Osborne and Patterson (2011) that characterizes a scientific explanation 
as a linguistic construct that contains three elements—a claim, evidence, and reasoning—as 
described in the previous section. 

Our LP for constructing explanations shows a progression from simple linguistic 
structures that include fewer elements, to more sophisticated structures that include more 
elements and connections among them. Levels 1 and 2 represent very poor explanatory structure 
that miss major components of the explanatory framework, such as evidence and reasoning. 
Levels 3 and 4 are intermediate models that present some causal relationships but lack coherence 
of reasoning with science principles. Levels 5 and 6 include sophisticated coherent models with 
all components of the explanatory framework, but differ in the extent of sufficient reasoning. 

Assessment Prototype Development Around the Competency Model and the Learning 

Progressions 

The CBAL science assessments are being developed based on the science competency 
model and its related hypothesized LPs. Below we elaborate several essential task features that 
support such alignment and present examples using a sample fonnative assessment prototype 
targeting the core idea of matter. This prototype is meant to measure and improve the teaching 
and learning of the matter core idea at Grade 6. In this formative assessment prototype, we are 
also interested in measuring students’ ability to engage in the practices of modeling and 
constructing scientific explanations. 
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Table 3 

The Learning Progression for Constructing Scientific Explanations 


Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 1: Nonstructural model 

• Student makes no claim or an inaccurate 

• Student does not make a claim 

• Make it explicit to students about what a 

claim. 

or makes an inaccurate claim. 

good claim is (i.e., an assertion or 


conclusion that addresses the original 
question or problem about a 
phenomenon). 

• Model and critique strengths and weakness 
of claims by showing examples. 

• Provide a rationale for creating claims. 

• Make use of students’ preexisting 
knowledge about claims. 

• Provide feedback in response to students’ 

_ claims. _ 

Level 2: Noncausal relation model 

• Student makes a claim. • Student does not back up the • Help students understand why it is 

Key words' claim claim with evidence. important to include justification to 

convince others to accept the claims. 

• Make it explicit to students what is 
evidence (i.e., the evidence supports the 
claim using scientific data). Data can 
come from an investigation that students 
complete or from another source, such as 
observations, reading material, or 
archived data, and needs to be both 
appropriate and sufficient to support the 
claim. 

• Model the justification of claims with 
examples. 

• Draw on what students know about 
evidence or justification in their everyday 
life and help them understand scientific 
evidence. 

• Provide feedback in response to students’ 
justification. 

Level 3: Insufficient causal relation model 

• Student makes an accurate claim. • The evidence is insufficient / • Make explicit to students what counts as 

• Student backs up the claim with evidence. inappropriate. appropriate and sufficient evidence (i.e., 

Key words: accurate claim, evidence b y appropriate, we mean data that are 

relevant to the problem and help support 
the claim; sufficient refers to providing 
enough data to convince another 
individual of the claim). Often providing 
sufficient evidence requires using multiple 
pieces of data. 

Model the justification of claims with 
sufficient evidence with examples. 

Draw on what students know about 
evidence or justification in their everyday 
life and help them understand what counts 
as good evidence. 

Provide feedback in response to students’ 
justifications. 
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Achievement 

Gap/challenge 

Instructional experience to support 
progression 

Level 4: Causal relation model 

• Student makes an accurate claim. 

• Student does not use reasoning 

• Help students understand why it is 

• Student backs up the claim with sufficient 

to tie the claim and evidence 

important to include reasoning to 

and appropriate evidence. 

together. 

convince others to accept the claims. 

Key words: accurate claim, appropriate and 
sufficient evidence 


• Make explicit to students what reasoning 
is (i.e., reasoning links the claim and 


evidence and shows why the data count as 
evidence to support the claim). 


• Model the use of reasoning with examples 
that tie claims to evidence. 

• Draw on what students know about 
reasoning in their everyday life and help 
them understand scientific reasoning. 

• Provide feedback in response to students’ 
reasoning. 


• Student makes an accurate claim. 

• Student backs up the claim with sufficient 
and appropriate evidence. 

• Student provides reasoning that links the 
claim and evidence and includes some, 
but not appropriate and sufficient, 
scientific principles. 

Key words: accurate claim, appropriate and 
sufficient evidence, reasoning _ 


Level 5: Insufficient coherence model 
• Student does not provide 
appropriate and sufficient 
scientific principles to link 
evidence and claims. 


Level 6: Sufficient coherence model 


• Make required components of a scientific 
explanation explicit to students. 

• Make explicit what good reasoning is in 
science. 

• Model and critique explanations. 

• Provide a rationale for creating 
explanations. 

• Connect to everyday explanations. 

• Assess and provide feedback to students. 


• Student constructs scientific explanation N/A N/A 

consisting of an accurate claim, sufficient 
and appropriate evidence, and sufficient 
reasoning including appropriate and 
sufficient scientific principles to tie the 
evidence to the claim. 


Key words: accurate claim, sufficient and 
appropriate evidence, sufficient reasoning 


We have used the hypothesized LP for matter to inform the design of our prototype task. 
In particular, we have used the LP to identify the mental model of matter that we would expect 
students to develop at early middle school, which also has implications for the kinds of 
phenomena that students can adequately explain and model. Our hypothesized LP suggests that 
students at the early middle school should be developing a particle model (see Table 1). This 
model includes tiny particles (although not necessarily atoms and molecules) that are in constant 
motion and that cannot be seen with the human eye. In this model, the type of particles, their 
motion, and their arrangement determine the macroscopic properties of the material that they 
make up. With this particle model, students should be able to predict, explain, and model the 
behavior of solids, liquids, and gases. Without a particle model, the behavior of gases or 
phenomena involving gases, such as evaporation, boiling, or condensation, cannot be well 
explained or predicted by students. For this reason, we have targeted phenomena that involve 
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evaporation and condensation in our prototype task. Many items in the exemplar formative 
assessment prototype are developed around the LP for the properties and structure of matter and 
the LP for change and conservation of matter. For example, some multiple-choice item options 
are aligned with different levels of the LPs to differentiate student understanding. In addition, we 
have developed items that require students to construct and describe graphic models of matter 
using a digital modeling tool (see Figure 5). We expect these items to elicit rich evidence of 
students’ initial and developing conceptions of matter addressed in our LP. These models of 
matter can be scored for presence of particles in student conceptions associated with the nature, 
arrangement, and behavior of those particles. For instance, we may find that some students’ 
models of matter stay at Level 1 or 2 during the course of the task and reflect only 
macroconceptions of matter; whereas, other students’ models may stay at, or move to, Level 3 
during the course of the task and include some particle representations along with some partially 
incorrect ideas (e.g., incorporating some macroscopic ideas and some missing elements of the 
behavior and arrangement of particles); and some students’ models may stay at, or move to, 
Level 4 or 5 during the course of the task and represent and describe particle ideas with more 
accuracies about the behavior and arrangement of particles. 

In addition, we have used our hypothesized LP for constructing scientific explanations to 
guide the design of the task. For example, our LP suggests that if students do not have sufficient 
knowledge about the structure of a good explanation, they may not be able to express what they 
know in an efficient way. Therefore, we provide a structure for students to enter the critical parts 
of a scientific explanation (i.e., their claim, evidence, and reasoning), in order to help them 
articulate their reasoning and, thus, help them externalize their mental model. 

In addition, we will use the LP for constructing scientific explanations to guide the 
scoring of students’ written scientific explanations collected from the task. The LP will allow us 
to anticipate and categorize the quality of students’ explanations as we collect them and assign 
them to particular levels in the LP. For example, in students’ written explanations, we expect a 
number of students to state a claim in their written explanations, but provide insufficient 
evidence (Level 3) or provide inappropriate and insufficient scientific principles (Level 4). 

In the following sections, we present several task features in the exemplar formative assessment 
protocol. In addition, we elaborate on the importance of including these features in our task 
design by connecting them with existing research in the learning sciences. 
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Scenario-Based Assessment 

Consistent with one of the key design principles underlying CBAL, we will create a 
reasonably realistic scenario for each assessment prototype. The literature in problem-based 
learning (PBL) research posits that engaging students in solving a real-world problem promotes 
knowledge acquisition, problem-solving skills, and motivation through authentic learning tasks 
(Hmelo & Lin, 2000). Thus, PBL, in turn, provides opportunities to measure the integrated 
application of both conceptual knowledge and science practices. In the scenario context, a 
driving question will be proposed in a conversational context that sets the goal for the whole 
task. In this exemplar formative task, students are asked to come up with an answer to the 
driving question: How can you get pure water to drink out of ocean water? We embedded the 
driving question in a context where three middle school students and their teacher (Ms. Jessie) 
are on a school trip at an ocean beach and are having the following conversation: 

Ms. Jessie: Hi, I am Ms. Jessie. Wow! What a wonderful day to spend with my 
students on a class field trip at an ocean beach! 

Tania: Hi, I am Tania. Ms. Jessie, I’m thirsty. I wish we could just drink the 

water in the ocean. There’s so much of it. 

Jack: Hi, I am Jack. Too bad that we can’t drink the ocean water. Can you 

imagine how much stuff there is in it? You can get really sick if you drink 
the ocean water. 

Ms. Jessie: You’re actually talking about a really important topic that affects many 

people and animals all around the world—getting enough pure water to 
drink every day. 

Locus: Hi, I am Locus. That’s what my dad told me. If scientists could only figure 

out a cheap and simple way to clean up ocean water to drink, a lot of 
people’s lives would be so much better and healthier. 

Ms. Jessie: Great! This is an excellent research idea. Why don’t we, as a small group, 
carry out an investigation and try to find a solution? The problem we need 
to solve is this: How can you get pure water out of ocean water? 
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We select this driving question because it is situated in a meaningful context for students and, 
therefore, should require and promote sense-making during the problem-solving process. In 
addition, it is a complex, open-ended problem that requires students to engage actively in 
analyzing and refining the problem an explaining, hypothesizing, representing, designing, and 
conducting investigations to test and revise hypotheses. Therefore, this scenario can provide 
many opportunities to measure both students’ conceptual understanding and their ability to apply 
science practices to solve problems. After the scenario is presented, the conversations among the 
student characters continue to help students refine the driving question into more manageable 
and testable questions, such as the following: What is in the pure water and the ocean water? 
How is pure water different from ocean water? To purify the ocean water, what should be left in 
and taken out? What process can help get the pure water back? Then various activities are 
designed to help students to respond to those refined questions. 

Simulation-Based Assessment 

Another feature of all CBAL science tasks is that we use simulations as opportunities to 
collect evidence of what students know and can do in science. By allowing students to interact 
with representations of phenomena, simulations provide an effective way for students to explore 
phenomena, or unseen mechanisms underlying phenomena, that would otherwise be impractical 
or difficult for students to observe in a typical assessment context. This expanded range of 
explorable content, in turn, allows assessments to measure students’ knowledge of content and 
science practices that are important to be tested, but which are not well assessed in traditional 
paper-and-pencil assessments. In addition, simulations allow us to create log data of students’ 
actions when working through a scientific investigation. Recent studies have shown that 
simulations can generate evidence of students’ achievement levels and measures of their science 
practices (Bennett, Persky, Weiss, & Jenkins, 2007; Quellmalz & Pellegrino, 2009). As 
Quellmalz, Timms, Silberglitt, and Buckley (2012) stated: “Because simulations are interactive, 
students can demonstrate their abilities to apply the active inquiry practices of science by 
designing investigations, conducting iterative trials, predicting, observing, and explaining 
findings, and critiquing the investigation of others” (p. 364). 

In this exemplar formative assessment prototype, the screenshot in Figure 2 displays an 
interactive simulation designed for students to manipulate and test predictions about the 
relationship between the number of particles and density of matter. Students can design multiple 
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experimental trials and explain why they think the experiments can test the provided prediction. 
For each trial, students will set up variables. After the students finish setting up the variables, a 
log data table will automatically record their setup variable values along with the values of the 
total volume, mass, and density of the liquid. Thus, students will be able to review a log of their 
interactions with the simulation tool and the outcome of their manipulation. The logged 
information will also be available for students later when they are asked to analyze the data and 
present evidence-based reasoning about whether the experiments support or refute the prediction. 
The simulation provides exploration opportunities for students to generate hypotheses about the 
relationship between mass, volume, and density in a pure material (water) and a mixture (filtered 
water). Then students can design experiments to test their hypotheses. By observing the particle 
interactions in the simulation, and analyzing the data patterns from the log data, students can 
collect evidence to support or reject their hypotheses. 



Figure 2. Screenshot of the formative assessment simulation prototype. 
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Scaffolded Formative Assessment 

Students benefit from scaffolds as they often construct knowledge in supported 
environments. Vygotsky’s (1978) “zone of a proximal development” model posits that the 
optimal task difficulty at which students should work for purposes of new learning is the one at 
which they can be successful, given appropriate instructional scaffolds. Process scaffolds can cue 
important components of inquiry (Duschl et ah, 2007). Our tasks will employ such scaffolds to 
provide an environment for students to work within their zones of proximal development. There 
are three characteristics of designed scaffolds in our fonnative assessment prototypes. First, 
scaffolding provides clear directions to explain what students must do to meet the expectations 
for a certain activity, thereby, reducing confusion and uncertainty. Second, scaffolding keeps 
purpose and motivation in the forefront by revealing to students why they are doing the activity. 
In scenario-based assessment, it is important to keep the big problem in focus. Third, scaffolding 
keeps students on task by providing a pathway and outlining the steps involved. This scaffolding 
requires the progression of activities to be liberating yet controlling at the same time. 

The screenshots in Figures 3 through 5 present an example of how we scaffold students 
to draw an initial model of ocean water, by providing explicit instruction on the key points of 
scientific models (Figure 3) and giving an example of a scientific model (Figure 4). Furthennore, 
as illustrated in Figure 5, we provide clear directions about what the student is supposed to 
model. In order to activate students’ prior knowledge and scaffold their thinking for the 
culminating performance, we provide prompts, such as what things need to be in the model and 
what things need not be in the model. Such scaffolds are important, since the epistemology of 
models receives little attention in normative and consensus views of the nature of science 
(Gobert & Pallant, 2004; Schwarz & White, 2005), in particular, using models to explain a 
natural phenomenon and using evidence to revise a model that fails to explain a phenomenon. In 
our task, we provide multiple opportunities for students to revisit their initial models and revise 
them after engaging in multiple activities. 
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Figure 3. Screenshot of the prototype—key points of modeling. 
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For example, a modern model of the solar system 
helps people understand and predict how the planets orbit 
around the Sun. There was a historical development 
process of our current solar system model. Over 2000 
years ago, Aristotle represented the Earth as the center of 
the solar system. This model was inaccurate because it 
could not explain and predict well how the Sun and 
planets appeared to move. Then over 450 years ago, 
Copernicus hypothesized a model with planets orbiting 
the Sun as the center of the solar system. Later, the 
modem solar system model supported Copernicus’s 
hypothesis but further revised the model based on his 
observations. This model better predicted the motion of 
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on the picture to explore the model. 
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Figure 4. Screenshot of the prototype—an example of modeling. 
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Figure 5. Screenshot of the prototype—drawing a model. 

Conclusions 

In this report, we present a middle school science competency model, three of its related 
learning progressions, and a fonnative assessment prototype developed under the guidance of the 
competency model and the learning progression. In this competency model and learning 
progressions, we attempt to describe how students represent knowledge and develop competence 
in the subject domain (Pellegrino et ah, 2001). The competency model and learning progressions 
begin with an understanding of the domain knowledge, science practices, and crosscutting 
concepts promoted by the NGSS (2013). In addition, we included epistemic beliefs and 
motivation, which are identified by the literature in science education and learning sciences as 
important influences. The hypothesized LPs allow us to design assessments that can, in principle, 
provide useful and understandable information to teachers, parents, and policy makers to guide 
their decision making. To illustrate how our competency model can be used to inform 
assessment development, we also present a sample fonnative assessment prototype. Through this 
example, we introduce several design features of the task that are consistent with our 
competency model and the research on how to support the teaching of science (such as the use of 
a rich problem-based scenario, the use of scaffolds, and the use of interactive simulations). 
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In the immediate future, we will apply our model of cognition to develop a set of 
formative and summative prototype assessment tasks and develop professional development 
materials to facilitate the implementation of the prototype assessments. The task development 
process will be iterative in nature and will involve the use of cognitive labs with students to test 
and refine the tasks and our model of cognition. Then we will conduct pilot studies to explore the 
construct validity of the prototype assessments, as well as the validity of the LPs. In addition to 
traditional psychometric approaches (e.g., multidimensional item-response models) to validation, 
we will also conduct cognitive psychological analyses, for example, analyzing what students do 
as they respond to the assessment task and what they say about task demands (Snow & Lohman, 
1989). 
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